Minimax Estimation of Divergences Between Discrete Distributions
نویسندگان
چکیده
منابع مشابه
Minimax Estimation of KL Divergence between Discrete Distributions
We refine the general methodology in [1] for the construction and analysis of essentially minimax estimators for a wide class of functionals of finite dimensional parameters, and elaborate on the case of discrete distributions with support size S comparable with the number of observations n. Specifically, we determine the “smooth” and “non-smooth” regimes based on the confidence set and the smo...
متن کاملVariational Minimax Estimation of Discrete Distributions under KL Loss
We develop a family of upper and lower bounds on the worst-case expected KL loss for estimating a discrete distribution on a finite numberm of points, given N i.i.d. samples. Our upper bounds are approximationtheoretic, similar to recent bounds for estimating discrete entropy; the lower bounds are Bayesian, based on averages of the KL loss under Dirichlet distributions. The upper bounds are con...
متن کاملMinimax Estimation of Discrete Distributions under $\ell_1$ Loss
We consider the problem of discrete distribution estimation under l1 loss. We provide tight upper and lower bounds on the maximum risk of the empirical distribution (the maximum likelihood estimator), and the minimax risk in regimes where the support size S may grow with the number of observations n. We show that among distributions with bounded entropy H , the asymptotic maximum risk for the e...
متن کاملMinimax Estimation of a Bounded Discrete Parameter
For a vast class of discrete model families with cdf’s Fθ, and for estimating θ under squared error loss under a constraint of the type θ ∈ [0,m], we present a general and unified development concerning the minimaxity of a boundary supported prior Bayes estimator. While the sufficient conditions obtained are of the expected form m ≤ m(F ), the approach presented leads, in many instances, to bot...
متن کاملEfficient MCMC estimation of discrete distributions
In this paper we propose an efficient Markov chain Monte Carlo (MCMC) method for estimation of discrete distributions by solving an appropriate system of linear equations. We call the estimator the equation-solving estimator. Our numerical results show that the new estimator makes significant improvements over the conventional frequencyMCMCestimator in terms of accuracy of the estimates. The ne...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Journal on Selected Areas in Information Theory
سال: 2020
ISSN: 2641-8770
DOI: 10.1109/jsait.2020.3041036